1. Field of the Invention
The present invention relates to a digital watermark technique for digital data, particularly image data.
2. Description of Related Art
Recently, from the viewpoint of copyright protection for digital data such as image data, etc., much attention is being paid to a digital watermark technique. The digital watermark technique is defined as a technique of embedding specific information into digital data according to a predetermined rule so that the specific information cannot be extracted from the digital data without using at least the predetermined rule. According to this technique, for example, information on a purchaser of image data, etc. is beforehand embedded in the image data itself according to the predetermined rule in such a manner that the information is not visible, and when the image data are illegally copied, the embedded information is extracted according to the predetermined rule from the data thus illegally copied to specify a person (purchaser) who illegally copies the data.
FIG. 18 is a diagram showing the principle of information embedding/extracting processing into/from image data on the basis of a conventional digital watermark technique.
With respect to the information embedding processing, as shown in FIG. 18, when a bit bi (0xe2x89xa6ixe2x89xa6n) of bits b1 to bn constituting information to be embedded is equal to 1, the bright of each of pixel data located at predetermined positions 1 to m of the image data is increased by U, and when the bit bi is equal to zero, the brightness of each pixel data is reduced by U. This processing is carried out on all the bits b1 to bn while the embedding position is shifted, thereby embedding the digital watermark information in the image data.
With respect to the processing of extracting the digital watermark information thus embedded, for the bit bi (0xe2x89xa6ixe2x89xa6n) of bits b1 to bn constituting the information thus embedded, the brightness sum S of all the pixel data located at the predetermined positions 1 to m of the image data and the brightness sum R (Reference value) of all the pixel data located adjacently to the predetermined positions 1 to m are detected. If Sxe2x88x92Rxe2x89xa7T (here, T is varied in accordance with a demanded error rate, and it is set to satisfy Txe2x89xa7Uxc3x97(the number m of pixel data in which bi is embedded)), it is judged that bi is equal to 1. If Sxe2x88x92Rxe2x89xa6xe2x88x92T, it is judged that bi is equal to zero. If xe2x88x92T less than Sxe2x88x92R less than T, it is judged that no information is embedded in each of the pixel data located at the predetermined positions 1 to m. This processing is carried out on all the bits b1 to bn to extract the digital watermark information in the image data.
The digital watermark technique is described in more detail in Kobayashi, et al.: Data Hiding Based on Neighbor Pixels Statistics, In Proc. of IPSJ 56th Annual Conference, 1V-03, (1998).
When a geometrical deformation such as scale-up/scale-down or rotation is applied to image data in which digital watermark information as described above is embedded, the embedding positions of the bit data constituting the digital watermark information (the positions on the X-Y coordinate used when the bit data are embedded) are also deformed. Therefore, the digital watermark information cannot be extracted. A technique to overcome the above problem is disclosed in U.S. Pat. No. 5,636,292.
According to this technique, as shown in FIG. 19, a calibrating digital watermark for estimating what geometrical deformation is applied to the image data, that is, what scale-up/scale-down or rotation processing is applied to the image data in addition to the digital watermark information such as copyright information or the like, is beforehand embedded in the image data concerned. When the digital watermark information is extracted, the calibrating digital watermark information is extracted from the image data, and the degree of the deformation is analyzed, thereby estimating the scale-up/scale-down degree or the rotational angle applied to the image data concerned. Subsequently, the image data concerned are returned to the original image data (the image data before the geometrical deformation is applied) by referring to the scale-up/scale-up rate or the rotational angle thus estimated. Thereafter, the pixel data located at a predetermined position and the adjacent pixel data are extracted, and the differential value (Sxe2x88x92R) therebetween is compared with a predetermined threshold value to extract the digital watermark information.
However, in the conventional method using the calibrating digital watermark as described above, not only digital watermark information such as copyright information, etc. which are to be originally embedded, but also a calibrating digital watermark are embedded in image data, and thus this method has a problem that a great effect is imposed on the image quality.
Therefore, a first object of the present invention is to provide a digital watermark technique which can specify a digital watermark information embedding position from geometrically-deformed image data without using any calibrating digital watermark.
Further, in the above conventional method of judging the bit value of the digital watermark information, for a bit bi (0xe2x89xa6ixe2x89xa6n) of bits b1 to bn constituting digital watermark information, the brightness sum S of all the pixel data located at the predetermined positions 1 to m of the image data and the brightness sum R (Reference value) of all the pixel data located adjacently to the predetermined positions 1 to m are detected. If Sxe2x88x92Rxe2x89xa7T, it is judged that bi=1. If Sxe2x88x92Rxe2x89xa6xe2x88x92T, it is judged that bi=zero. If xe2x88x92T less than Sxe2x88x92R less than T, it is judged that no information is embedded in each of the pixel data located at the predetermined positions 1 to m.
Here, a predetermined value is usually used as the threshold value T. However, when the threshold value T is not optimized to the image data from which the digital watermark information is extracted, xe2x88x92T less than Sxe2x88x92R less than T is established although the bit bi constituting the digital watermark information is embedded to predetermined positions 1 to m of the image data, and thus it may be judged that no information is embedded in the predetermined positions 1 to m. That is, the extraction sensitivity of the bits constituting the digital watermark information is lowered.
On the other hand, if the extraction sensitivity of the bits constituting the digital watermark information is increased by setting the threshold value T to a lower value in advance, for some image data from which the digital watermark information is extracted, the bit value judgment error of the bits constituting the digital watermark information extracted from the image data is increased, so that the bit value judgment precision is degraded. Therefore, the bit value of the digital watermark information extracted from the image data cannot be relied on.
Therefore, a second object of the present invention is to provide a digital watermark technique which can grasp whether the bit value of digital watermark information extracted can be relied on even when the extraction sensitivity of the bits constituting the digital watermark information is increased.
In order to attain the first object, according to a first aspect of the present invention, a method of extracting digital watermark information from image data which has the digital watermark information embedded therein by altering at least one pixel data located at a predetermined position on a specific coordinate and is geometrically deformed, comprises an embedding position check step of executing the processing of extracting at least one pixel data at a predetermined position on the specific coordinate from the image data and comparing the data value thus extracted with a reference value to judge whether the information is embedded in the pixel data while applying the geometrical deformation on the image data until it is confirmed that the information is embedded in the pixel data.
Here, the geometrical deformation means a reversible deformation such as scale-up/scale-down or rotation, that is, it means that deformed data can be restored to original image data before the deformation. Further, the reference value may be the data value of pixel data located in the vicinity of the extracted pixel data or the data value of the extracted pixel data which is interpolatively estimated by using at least two pixel data located in the vicinity of the extracted pixel data. Further, xe2x80x9cuntil it is checked that the information is embedded in the pixel dataxe2x80x9d specifically means xe2x80x9cuntil the image data which is subjected to the geometrical deformation so that the differential value (absolute value) between the data value of the extracted pixel data and the reference value is maximum is detectedxe2x80x9d.
According to the first aspect of the present invention, a series of processing of extracting from the image data the pixel data located at the embedding position specified by a specific coordinate and comparing the data value of the pixel data concerned with the reference value to judge whether the information is embedded in the pixel data concerned is executed while applying the geometrical deformation on the image data until it is checked that the information is embedded in the pixel data. For example, the image data is rotated at a rotational angle, and then, it is checked whether the information is embedded in the pixel data while further the image data concerned is scaled up/scaled down by a predetermined magnification. If it is not checked that the information is embedded in the pixel data, the image data is rotated at another rotational angle and then the same processing is carried out. This processing is repetitively executed until it is checked that the information is embedded in the pixel data.
With the above operation, without the calibrating digital watermark, the position at which the digital watermark information is embedded can be specified from the geometrically-deformed image data.
In the first aspect of the present invention, the embedding position check step may comprise the following roughly checking step and detailed checking step. In the roughly checking step, the processing of extracting from the image data at least one pixel data located at a predetermined position on the specific coordinate and comparing the data value thus extracted with the reference value to judge whether the information is embedded in the pixel data concerned is carried out while the geometrical deformation is applied to the image data by every first geometrical deformation rate until it is confirmed that the information is embedded in the pixel data concerned. In the detailed checking step, the processing of extracting from the image data at least one pixel data located at a predetermined position on the specific coordinate and comparing the data value thus extracted with the reference value to judge whether the information is embedded in the pixel data concerned is carried out while the geometrical deformation is applied to the image data by every second geometrical deformation rate smaller than the first geometrical deformation rate within a predetermined range containing the geometrical deformation rate when it is checked in the roughly checking step that the information is embedded in the pixel data until it is confirmed that the information is embedded in the pixel data concerned.
Further, in the first aspect of the present invention, when the digital watermark information is embedded in the image data by altering at least one pixel data located at a predetermined position on a specific coordinate for each of plural bit data constituting the digital watermark information, the embedding position check step may be carried out on at least one bit data of the plural bit data, whereby the plural bit data are extracted from the image data which are subjected to the geometrical deformation when the pixel data having the bit data concerned embedded therein is confirmed.
With the above operation, the check of the pixel data having the digital watermark information embedded therein can be more efficiently performed.
In order to attain the first object, according to a second aspect of the present invention, a method of extracting digital watermark information from image data which has the digital watermark information embedded therein by altering at least one pixel data located at a predetermined position on a specific coordinate and is geometrically deformed, comprises an embedding position check step of executing the processing of extracting at least one pixel data located at a predetermined position on the specific coordinate and comparing the data value thereof with a reference value to judge whether the information is embedded in the pixel data concerned while varying the predetermined position on the image data by deforming the specific coordinate until it is confirmed that the information is embedded in the pixel data concerned.
According to the second aspect of the present invention, a series of processing of extracting from the image data the pixel data located at the embedding position of the digital watermark information specified by the specific coordinate and then comparing the data value of the pixel data concerned with the reference value to judge whether the information is embedded in the pixel data concerned is executed while deforming the specific coordinate until it is confirmed that the information is embedded in the pixel data. For example, the specific coordinate is rotated at a rotational angle, and then, it is checked whether the information is embedded in the pixel data while scaled up/scaled down by a predetermined magnification. If the embedding of the information is not confirmed, the specific coordinate is rotated at another rotational angle, and then, the same processing is carried out. This processing is repetitively carried out until it is confirmed that the information is embedded in the pixel data.
With the above operation, the digital watermark information can be extracted from the geometrically-deformed image data without using any calibrating digital watermark. Further, according to the second aspect of the present invention, the embedding position of the digital watermark information can be checked without applying any geometrical deformation on the image data itself, so that the memory capacity needed for the processing and the load of the processor can be reduced to be less than these of the first aspect of the present invention.
Further, in the second aspect of the present invention, the embedding position check step may comprise the following roughly checking step and detailed checking step. In the roughly checking step, the processing of extracting from the image data at least one pixel data located at a predetermined position on the specific coordinate and comparing the data value thus extracted with the reference value to judge whether the information is embedded in the pixel data concerned is carried out while the predetermined position on the image data is varied by deforming the specific coordinate by every first deformation rate until it is confirmed that the information is embedded in the pixel data concerned. In the detailed checking step, the processing of extracting from the image data at least one pixel data located at a predetermined position on the specific coordinate and comparing the data value thus extracted with the reference value to judge whether the information is embedded in the pixel data concerned is carried out while the predetermined position on the image data is varied by deforming the specific coordinate by every second deformation rate smaller than the first deformation rate within a predetermined range containing the deformation rate when it is confirmed in the roughly checking step that the information is embedded in the pixel data until it is confirmed that the information is embedded in the pixel data concerned.
The first and the second deformation rate is determined, in some cases, according to the embedding positions in the image data of the watermark information, namely, a pattern of the watermark. With some pattern, the embedded watermark information can be detected, even when the rate of the deformation given to the image data having the watermark embedded therein is not the same as the deformation rate of the image data or the deformation rate of the coordinate for detecting the embedded information from the image data. In such a case, even when the first and the second deformation rate is made larger, there may still be a high possibility that the embedded information is detected from the image data. Or, when the first deformation rate is made larger and the second deformation rate is made smaller, there may still be a high possibility that the embedded information is detected from the image data.
Therefore, in the case where the alteration of the pixel data effected due to the embedding of the watermark information is made by a unit of a pixel block larger than 1xc3x971 and the roughly checking step is conducted such that one of the pixels of the pixel block is detected, the first deformation rate can be larger.
FIGS. 20 and 21 are schematic diagram showing the positional relationship between the positions in the image data where pixel data are altered due to the embedded information and the detecting positions to detect pixel data concerned which are determined by the coordinate deformation for detection. The alteration of the pixel data due to the embedded information is made by a block unit of 2xc3x972 at 24 positions with a pitch of 10 pixels. In FIG. 20, there is no deviation between the positions where the altered pixel data are located and the detecting positions to detect the altered pixel data. All the 24 detecting positions overlap with the positions subjected to the alteration, respectively. In this case, the embedded information can be detected correctly. On the other hand, in FIG. 21, there is a deviation between the alteration positions in the image data and the detection positions for the image data. In total, there is a deviation of two pixels in the length of 240 pixels. FIG. 21 shows the case where the detection coordinate is scaled up by 242/240=1.0083 with respect to the image data. When the deformation rate is increased to that scale, the embedded information cannot be detected at one of the 24 positions and detection failure is caused. To solve this problem, the scale or the deformation rate may be lowered to less than 1.0083. When the size of the pixel block subjected to the alteration is enlarged from 2xc3x972 to 4xc3x974, the deformation rate may be as rough as 244/240=1.0167, still ensuring the detection of the embedded information.
More specifically, when the size of the pixel block is selected to be Pxc3x97P and N positions are subjected to the alteration with a pitch of D pixels, the first deformation rate may be increased upto (DN+P)/(DN)=1+P/(DN), still ensuring the detection of the embedded information in the roughly checking step.
Although the detection in terms of coordinate scaling-up/down is mentioned as an example in the above, the coordinate rotation or the scaling-up/down or rotation of the image data itself can also be employed.
In the second aspect of the present invention, when the digital watermark information is embedded in the image data by altering at least one pixel data located at a predetermined position on a specific coordinate for each of plural bit data constituting the digital watermark information, the embedding position check step may be carried out on at least one bit data of the plural bit data, whereby the plural bit data are extracted by using the deformation-applied specific coordinate when the pixel data having the bit data concerned embedded therein is confirmed.
With the above operation, the check of the embedding position of the digital watermark information can be more efficiently performed.
Further, in the second aspect of the present invention, it may be adopted that when the digital watermark information is embedded in the image data by altering at least one pixel data located at a predetermined position on a specific coordinate for each of plural bit data constituting the digital watermark information, the embedding position checking step is carried out on at least one bit data of the plural bit data, a geometrical deformation is applied to the image data at the geometrical deformation rate corresponding to the deformation rate applied to the specific coordinate when the pixel data having the bit data concerned embedded therein is confirmed, and the plural bit data are extracted from the geometrically-deformed image data by using the specific coordinate before the deformation in the embedding position checking step.
Usually, when image data are scaled down, any pixel data is removed from the image data according to the scale-down rate of the image data to reduce the size of the image data. Therefore, when the scale-down processing is carried out on image data having digital watermark information embedded therein, pixel data having (a part of) the digital watermark information may be removed from the image data concerned in the scale-down processing. In such a case, the overall digital watermark information may not be efficiently extracted from the image data concerned.
Therefore, as described above, the embedding position checking step is carried out on at least one bit data of plural bit data constituting the digital watermark information to check the pixel data having the bit data concerned embedded therein. If the pixel data is confirmed, the image data are scaled up with the scale-up rate corresponding to the scale-down rate of the coordinate when the pixel data is confirmed, thereby returning the size of the image data to the original size (the size before the scale-down processing is carried out). Accordingly, pixel data which are lost due to the scale-down processing are restored and the plural bit data constituting the digital watermark information are extracted to thereby enhance the detection precision.
In order to attain the second object, according to a third aspect of the present invention, a bit value judgment method of judging a bit value of digital watermark information embedded in image data in which the digital watermark information is embedded by altering at least one pixel data located at a predetermined position on a specific coordinate for each of plural bit data constituting the digital watermark information, comprises: an error rate calculation step of calculating, for each of at least two bit data of plural bit data constituting the digital watermark information, the differential value between pixel data in which the bit data concerned is to be embedded and a reference value, and determining, on the basis of a probability distribution curve settled by the distribution of the differential values thus calculated, a bit value judgement error rate of the plural bit data when a threshold value used for the bit value judgment of the bit data is represented by T; and a bit value judgment step of comparing, for each of the plural bit data constituting the digital watermark information, the threshold value T and the differential value between pixel data in which the bit data concerned is to be embedded and the reference value when the judgment error rate calculated in the error rate calculation step is smaller than a predetermined value, thereby judging the bit value (0 or 1) of the bit data concerned.
Here, the probability distribution curve settled by the distribution of the differential values may be calculated from a statistic value such as the average value or dispersion value of the differential values obtained for each of at least the two bit data of the plural bit data constituting the digital watermark information, or a histogram of the differential values.
The bit value judgment error rate of the bit data concerned when the threshold value used for the bit value judgment of the bit data is represented by T may be calculated as the occupational rate of a part located within the range from xe2x88x92T to T to the overall probability distribution curve settled by the distribution of the differential values.
According to the third aspect of the present invention, prior to the bit value judgment of the digital watermark information embedded in the image data, the bit value judgment error rate of the digital watermark information embedded in the image data concerned when the threshold value used for the bit value judgment is represented by T is calculated, and only when the calculation result is smaller than a predetermined value, the bit value judgment of the digital watermark information embedded in the image data concerned is carried out by using the threshold value T. Therefore, even when the threshold value T used for the bit value judgment is set to a low value to increase the extraction sensitivity of the bits constituting the digital watermark information, the reliability of the bit value of the digital watermark information thus extracted can be ensured.
Further, in order to attain the second object, according to a fourth aspect of the present invention, a bit value judgment method of judging a bit value of digital watermark information embedded in image data in which the digital watermark information is embedded by altering at least one pixel data located at a predetermined position on a specific coordinate for each of plural bit data constituting digital watermark information, comprises: an error rate calculation step of calculating, for each of at least two bit data of plural bit data constituting the digital watermark information, the differential value between pixel data in which the bit data concerned is to be embedded and a reference value, and determining, on the basis of a probability distribution curve settled by the distribution of the differential values thus calculated, a bit value judgement error rate of the plural bit data when a threshold value used for the bit value judgment of the bit data is represented by T; a bit value judgement step of comparing, for each of the plural bit data constituting the digital watermark information, the threshold value T with the differential value between pixel data in which the bit data concerned is to be embedded and the reference value, thereby judging the bit value (0 or 1) of the bit data concerned; and an output step of outputting the bit values of the plural bit data constituting the digital watermark information judged in the bit value judgment step together with the bit value judgment error rate calculated in the error rate calculation step.
According to the fourth aspect of the present invention, the bit value judgment result of the digital watermark information embedded in the image data is output together with the bit value judgment error rate for the threshold value T used for the bit value judgment of the digital watermark information embedded in the image data. Accordingly, when the threshold value T used for the bit value judgment is set to a low value to increase the extraction sensitivity of the bits constituting the digital watermark information, it can be grasped how degree the reliability of the bit value of the digital watermark information thus extracted is.
In the third and fourth aspects of the present invention, when the threshold value T is set to zero, in the bit value judgment of the digital watermark information, by checking, for each of the plural bit data constituting the digital watermark information, whether the differential value between pixel data in which the bit data concerned is to be embedded therein and the reference value is positive or negative, the bit value (0 or 1) of the bit data concerned is judged. That is, by setting the threshold value T for the bit value judgment to zero, the judgment of 0 or 1 is necessarily made for each of the plural bit data constituting the digital watermark information embedded in the image data, thereby maximizing the extraction sensitivity of the bits constituting the digital watermark information.
In this case, in the error judgment step, for each of at least two bit data of the plural bit data constituting the digital watermark information, the differential value between the pixel data in which the bit data concerned is to be embedded and the reference value is calculated, and at least two differential values thus calculated are classified into two subsets on the basis of the judgement as to whether the differential value is positive or negative, and two probability distribution curves are determined on the basis of the distribution of the differential values contained in each of the two subsets. The bit value judgment error rate can be calculated by checking the rate of the negative part in the probability distribution curve of the positive subset and/or the rate of the positive part in the probability distribution curve of the negative subset.
In order to attain the first and second objects, according to a fifth aspect of the present invention, the digital watermark extraction method of the first or second aspect of the present invention is combined with the bit value judgment method of the digital watermark information of the third or fourth aspect of the present invention. That is, the embedding position of each bit constituting the digital watermark information embedded in the image data is specified by the digital watermark extraction method of the first or second aspect, and the bit value judgment of the digital watermark information is carried out on the image data for which the embedding position of each bit constituting the digital watermark information as described above is specified, by the bit value judgment method of the digital watermark information of the third or fourth aspect of the present invention.
Here, a device for executing the digital watermark extraction method of the first or second aspect and a device for executing the bit value judgment method of the digital watermark information of the third or fourth aspect may be the same, or may be separate and independent devices.
The error rate judgment step of the third to fifth aspects of the present invention may be applied to a case where it is merely judged whether watermark information is embedded in image data. That is, according to a sixth aspect of the present invention, a digital-watermark-information presence or absence judging method of judging whether digital watermark information is embedded in image data by altering at least one pixel data located at a predetermined position on a specific coordinate in the image data for each of plural bit data constituting the digital watermark information, comprises a step of calculating, for each of at least two bit data of plural bit data constituting the digital watermark information, the differential value between pixel data in which the bit data concerned is to be embedded and a reference value, and a step of judging it on the basis of a probability distribution curve settled by the distribution of the differential values thus calculated whether the digital watermark information is embedded.